AutoML Leaderboard
| Best model | name | model_type | metric_type | metric_value | train_time |
|---|---|---|---|---|---|
| 1_Baseline | Baseline | average_precision | 0.504348 | 1.57 | |
| 2_DecisionTree | Decision Tree | average_precision | 0.9227 | 3.02 | |
| 3_Linear | Linear | average_precision | 0.986073 | 3.88 | |
| the best | 4_Default_Xgboost | Xgboost | average_precision | 1 | 4.43 |
| 5_Default_NeuralNetwork | Neural Network | average_precision | 0.992316 | 2.77 | |
| 6_Default_RandomForest | Random Forest | average_precision | 0.986895 | 3.49 | |
| Ensemble | Ensemble | average_precision | 1 | 1.56 |
AutoML Performance
AutoML Performance Boxplot
Features Importance
Spearman Correlation of Models
Summary of 1_Baseline
Baseline Classifier (Baseline)
- n_jobs: -1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
average_precision
Training time
1.1 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.693177 | nan |
| auc | 0.5 | nan |
| f1 | 0.67052 | 0.448688 |
| accuracy | 0.504348 | 0.448688 |
| precision | 0.504348 | 0.448688 |
| recall | 1 | 0.448688 |
| mcc | 0 | 0.448688 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.693177 | nan |
| auc | 0.5 | nan |
| f1 | 0.67052 | 0.448688 |
| accuracy | 0.504348 | 0.448688 |
| precision | 0.504348 | 0.448688 |
| recall | 1 | 0.448688 |
| mcc | 0 | 0.448688 |
Confusion matrix (at threshold=0.448688)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 0 | 57 |
| Labeled as 1 | 0 | 58 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 2_DecisionTree
Decision Tree
- n_jobs: -1
- criterion: gini
- max_depth: 3
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
average_precision
Training time
2.5 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.31362 | nan |
| auc | 0.924682 | nan |
| f1 | 0.877193 | 0.393939 |
| accuracy | 0.878261 | 0.393939 |
| precision | 1 | 0.85 |
| recall | 1 | 0.0782609 |
| mcc | 0.757035 | 0.393939 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.31362 | nan |
| auc | 0.924682 | nan |
| f1 | 0.877193 | 0.393939 |
| accuracy | 0.878261 | 0.393939 |
| precision | 0.892857 | 0.393939 |
| recall | 0.862069 | 0.393939 |
| mcc | 0.757035 | 0.393939 |
Confusion matrix (at threshold=0.393939)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 51 | 6 |
| Labeled as 1 | 8 | 50 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 3_Linear
Logistic Regression (Linear)
- n_jobs: -1
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
average_precision
Training time
3.1 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.163214 | nan |
| auc | 0.980944 | nan |
| f1 | 0.948276 | 0.41277 |
| accuracy | 0.947826 | 0.41277 |
| precision | 1 | 0.568079 |
| recall | 1 | 0.00812168 |
| mcc | 0.900647 | 0.568079 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.163214 | nan |
| auc | 0.980944 | nan |
| f1 | 0.948276 | 0.41277 |
| accuracy | 0.947826 | 0.41277 |
| precision | 0.948276 | 0.41277 |
| recall | 0.948276 | 0.41277 |
| mcc | 0.895644 | 0.41277 |
Confusion matrix (at threshold=0.41277)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 54 | 3 |
| Labeled as 1 | 3 | 55 |
Learning curves
Coefficients
| feature | Learner_1 |
|---|---|
| PEER_PRESSURE | 1.90678 |
| YELLOW_FINGERS | 1.73776 |
| CHRONIC DISEASE | 1.69633 |
| ALLERGY | 1.68397 |
| SWALLOWING DIFFICULTY | 1.65569 |
| COUGHING | 1.60986 |
| ALCOHOL CONSUMING | 1.59961 |
| FATIGUE | 1.27991 |
| WHEEZING | 1.19506 |
| ANXIETY | 0.865616 |
| SMOKING | 0.36864 |
| CHEST PAIN | 0.244263 |
| SHORTNESS OF BREATH | 0.108942 |
| AGE | 0.10044 |
| GENDER | -0.0231852 |
| intercept | -4.98493 |
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
SHAP Importance
SHAP Dependence plots
Dependence (Fold 1)
SHAP Decision plots
Top-10 Worst decisions for class 0 (Fold 1)
Top-10 Best decisions for class 0 (Fold 1)
Top-10 Worst decisions for class 1 (Fold 1)
Top-10 Best decisions for class 1 (Fold 1)
Summary of 4_Default_Xgboost
Extreme Gradient Boosting (Xgboost)
- n_jobs: -1
- objective: binary:logistic
- eta: 0.075
- max_depth: 6
- min_child_weight: 1
- subsample: 1.0
- colsample_bytree: 1.0
- eval_metric: average_precision
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
average_precision
Training time
3.7 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.144457 | nan |
| auc | 1 | nan |
| f1 | 1 | 0.296689 |
| accuracy | 1 | 0.296689 |
| precision | 1 | 0.296689 |
| recall | 1 | 0.0426503 |
| mcc | 1 | 0.296689 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.144457 | nan |
| auc | 1 | nan |
| f1 | 1 | 0.296689 |
| accuracy | 1 | 0.296689 |
| precision | 1 | 0.296689 |
| recall | 1 | 0.296689 |
| mcc | 1 | 0.296689 |
Confusion matrix (at threshold=0.296689)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 0 | 58 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
SHAP Importance
SHAP Dependence plots
Dependence (Fold 1)
SHAP Decision plots
Top-10 Worst decisions for class 0 (Fold 1)
Top-10 Best decisions for class 0 (Fold 1)
Top-10 Worst decisions for class 1 (Fold 1)
Top-10 Best decisions for class 1 (Fold 1)
Summary of 5_Default_NeuralNetwork
Neural Network
- n_jobs: -1
- dense_1_size: 32
- dense_2_size: 16
- learning_rate: 0.05
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
average_precision
Training time
2.2 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.121825 | nan |
| auc | 0.990926 | nan |
| f1 | 0.973451 | 0.593571 |
| accuracy | 0.973913 | 0.593571 |
| precision | 1 | 0.593571 |
| recall | 1 | 0.000153975 |
| mcc | 0.949138 | 0.593571 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.121825 | nan |
| auc | 0.990926 | nan |
| f1 | 0.973451 | 0.593571 |
| accuracy | 0.973913 | 0.593571 |
| precision | 1 | 0.593571 |
| recall | 0.948276 | 0.593571 |
| mcc | 0.949138 | 0.593571 |
Confusion matrix (at threshold=0.593571)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 3 | 55 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of 6_Default_RandomForest
Random Forest
- n_jobs: -1
- criterion: gini
- max_features: 0.9
- min_samples_split: 30
- max_depth: 4
- eval_metric_name: average_precision
- explain_level: 2
Validation
- validation_type: split
- train_ratio: 0.75
- shuffle: True
- stratify: True
Optimized metric
average_precision
Training time
2.9 seconds
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.203226 | nan |
| auc | 0.984876 | nan |
| f1 | 0.945455 | 0.585946 |
| accuracy | 0.947826 | 0.585946 |
| precision | 1 | 0.585946 |
| recall | 1 | 0.0241343 |
| mcc | 0.900647 | 0.585946 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.203226 | nan |
| auc | 0.984876 | nan |
| f1 | 0.945455 | 0.585946 |
| accuracy | 0.947826 | 0.585946 |
| precision | 1 | 0.585946 |
| recall | 0.896552 | 0.585946 |
| mcc | 0.900647 | 0.585946 |
Confusion matrix (at threshold=0.585946)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 6 | 52 |
Learning curves
Permutation-based Importance
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve
Summary of Ensemble
Ensemble structure
| Model | Weight |
|---|---|
| 4_Default_Xgboost | 1 |
Metric details
| score | threshold | |
|---|---|---|
| logloss | 0.144457 | nan |
| auc | 1 | nan |
| f1 | 1 | 0.296689 |
| accuracy | 1 | 0.296689 |
| precision | 1 | 0.296689 |
| recall | 1 | 0.0426503 |
| mcc | 1 | 0.296689 |
Metric details with threshold from accuracy metric
| score | threshold | |
|---|---|---|
| logloss | 0.144457 | nan |
| auc | 1 | nan |
| f1 | 1 | 0.296689 |
| accuracy | 1 | 0.296689 |
| precision | 1 | 0.296689 |
| recall | 1 | 0.296689 |
| mcc | 1 | 0.296689 |
Confusion matrix (at threshold=0.296689)
| Predicted as 0 | Predicted as 1 | |
|---|---|---|
| Labeled as 0 | 57 | 0 |
| Labeled as 1 | 0 | 58 |
Learning curves
Confusion Matrix
Normalized Confusion Matrix
ROC Curve
Kolmogorov-Smirnov Statistic
Precision-Recall Curve
Calibration Curve
Cumulative Gains Curve
Lift Curve